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ABSTRACT 

An approach to educational research that is receiving 
renewed attention is participatory action research (PAR), which 
emphasizes a bottom-up approach to research as well as useful 
outcomes. PAR is a philosophy, and not a methodology per se. It is a 
deliberate, goal-directed, structured, and applied means of obtaining 
information that looks to informants as active participants in the 
evaluation and research process. The general izabi 1 ity of findings is 
deemphasized in favor of localized, individual ly meaningful action 
outcomes. PAR emerges from sociology and soci il/cultural 
anthropology, and can use mixed methods and any design in which 
evaluat ion/research is participant directed. Several examples are 
given of the PAR approach. Appendix A describes a program evaluation 
conducted at the Department of Science and Engineering Support at the 
National Technical Institute for the Deaf (NTID) of the Rochester 
(New York) Institute of Technology involving 56 students. The survey 
instrument is included. Appendix B describes an interview study in 
which staff at the Instructional Design and Evaluation Department of 
the NTID were asked about the quality of instructional materials and 
the data were used to create guidelines and checklists for project 
planning and development. Appendix C contains a report on the 
development of instructional materials for two courses at the NTID. 
(SLD) 
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There has recently been a confluence of several trends that suggest some new 
attention be given to our old ways of approaching educational research. One of these 
trends is a heightened tension between pure and applied science. Another is 
consumerism — i.e. consumer-driven or consumer-responsive research, programming, 
etc.). Third, the detached, analytical researcher studying human relations from the 
laboratory is rapidly becoming out of favor. New ways are being called for and 
educational researchers are rising to the challenge. 

One approach receiving renewed attention is participatory action research or 
PAR (as in being on equal par). While two earlier Division D presentations (Scherer 
& McKee, 1991; 1992) discussed the results and outcomes of PAR efforts, PAR itself 
was not the focus of the presentations. 

PAR emphasizes a bottom-up approach to research and emphasizes useful 
outcomes. According to W.F. Whyte, "It is important, both for the advancement of 
science and for the improvement of human welfare, to devise strategies in which 
research and action are closely linked'' (1991, p. 8). 
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What PAR Is 

PAR is philosophy, and not a methodology per se. It is a deliberate, goal- 
directed, structured and applied means of obtaining information — one that looks to 
informants not as passive providers of information but as active participants in the 
evaluation/research process. PAR de-emphasizes the "generalizeability of findings" in 
favor of localized, individually meaningful action outcomes. 

The current interest in PAR derives in part from a desire to move away from the 
"professional expert" and "technology transfer" models. Whyte uses the examples of 
traditional organizational research and agricultural technology diffusion respectively 
to illustrate the value of a more participant-directed approach. 

What PAR Isn't 

PAR is not another name for a heuristic approach or a license for a "fishing 
expedition." While emerging from sociology and social/cultural anthropology, it need 
not be solely ethnographic or qualitative and can employ mixed methods. Any design 
can be used as long as the evaluation/research is participant-directed (key informants 
serve as collaborators) and action outcome-driven. 

How it's Done 

While closely related to the technique "participant observation," other 
techniques used include study circles, focus groups, Delphi-like techniques, and one- 
on-one or group interviews. Our research illustrates its use in "Matching Persons with 
Technologies" and for a postsecondary institution for deaf students. 



Some examples 

1. There is considerable concern that assistive and educational technologies 
are being widely recommended without full consideration being given to 
individuals' predispositions ';o their use. The desired outcome is 
heightened attention to the psychosocial aspects of technology use. 
Assessment instruments have been created from the actual experiences 
of people who profiled themselves as users or non-users of particular 
assistive or educational technologies. The PAR approach and methods 
used to develop the instruments have been described in Scherer 
(1991a,b,c) and Scherer & McKee (1991). When collecting data on the 
instruments' reliability and validity, a PAR approach was used again. 

2. The Department of Science and Engineering Support at NTID requested 
a program evaluation. A sample of faculty and students were 
interviewed and the data was used to create a questionnaire that was 
distributed to all students. The full report of this multiphase project is 
in Appendix A. 

3. All staff in NTID's Instructional Design and Evaluation department were 
interviewed about the criteria used in judging the quality of instructional 
materials (the results are in Appendix B). '. n he data were used to create 
guidelines/checklists for project planning and development. These 
guidelines are being used on a continuous basis. 



4. Instructional materials needed to be developed for two courses in NTID's 
Optical Finishing Technology major. Former and current students were 
given prototype versions of the materials and were asked to participate 
in discussions regarding modifications to the materials. The full report 
of these efforts is in Appendix C. 

What You Get 

The results of the above, as with most PAR efforts, led to consumer-identified 
changes/modifications. In these examples, the modifications were in a) assessment 
instruments, b) course content and c) instructional delivery systems. 
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The Department of Science and Engineering Support (DSES): 
A Phase 2 Report of Student Perceptions 

Introduction 

A multi-phase evaluation of the Department of Science and 
Engineering Support was initiated during the 1986-87 academic 
year. The first phase involved the identification of issues of 
current importance to a) DSES major and service students, b) 
RIT science and engineering faculty, and c) faculty members of 
DSES. Intensive, open-ended interviews with eleven cross- 
registered students (5 majors and 6 service students), five RIT 
faculty members, and all members of DSES resulted in the 
identification of many issues that were subsequently 
incorporated into a questionnaire circulated to all DSES major 
and service students towards the end of the spring 1988 
quarter. 

The results of the first phase (hereafter referred to as 
Phase 1) interviews were presented in a report (Scherer & 
Binder, 1987) circulated only among the faculty members of DSES 
who were asked to read the report and identify points they 
would like to explore further in a questionnaire. Their 
feedback during subsequent discussions provided the basis for a 
twenty- item, 8-page, questionnaire (see Appendix) that is 
herein referred to as Phase 2. 

All students receiving support services from DSES during 
the spring 1988 quarter were asked to complete the 



questionnaire and to sign-up for a half-hour small group 
administration of the questionnaire. 

Fifty-six questionnaires were completed, which represents 
38% of the total eligible population (76% of eligible majors 
responded and 19% of eligible service students responded) . 
Respondents had been in a cross-registered program from 1 to 6 
years, with the mean length of time being 2.4 years. Thus, 
there is an over-representation of science and engineering 
majors as compared to service students. 

Respondents were taking a mean of 2 . 1 science/math/ 
engineering courses spring 1988 quarter (range=0-6). 

Student responses to the questionnaire items are described 
by topic area. First, the most salient information from the 
Phase 1 interviews is presented. Ser^nd, the data is presented 
and discussed, and third, responses to the open-ended questions 
are summarized with selected quotes serving as illustrations. 



Tutoring 

During the Phase 1 interviews students expressed a general 
preference for individual tutoring rather than group tutoring. 
The stated reasons were the communication differences among 
students in the groups and the possibility that one student 
will monopolize the group. They wanted to augment group 
tutoring with individual tutoring. Students also preferred 
professional tutoring to peer student tutoring because the 
professional tutors know how deaf students learn best and they 



organize and present the material most effectively. Newer 
students didn't take as much advantage of tutoring as older 
students. Some students have personal learning styles that 
they said interfered with the tutoring process. The need to 
feel independent and being embarrassed because they do not 
understand the course content kept some students from seeking 
tutoring, 

Phase 2 

pnp.fition 1. How many times do vou usually OO for tutoring 
^n science/math/e n gineerin g courses?. A year later, when 
students responded to the questionnaire, the distribution of 
the mean responses for this group of students across all 
tutoring options indicates that individual tutoring with a 
faculty member was the most frequently used tutoring option 
(see Figure 1). Group tutoring with a student peer tutor was 
the least frequently used option. Frequency of use of faculty 
tutors was very different from that reported for use of peer 
student tutors. As can be seen in Figure 1, the most 

frequently used tutoring option is "individual with faculty." 
69.1% responded that they use this option at least once a week. 
When asked about their use of group tutoring, 32.7% responded 
that they use group tutoring with support faculty members at 
least once a week. Alternately, 27.3% use indivf \ual tutoring 
with a peer student tutor once a week or more and 13.0% used 
group tutoring with a peer once a week or more. 

The respondents were then divided into two categories: a) 
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student type as being either a science major, engineering 
major, or service student, and b) first year versus older 
students. When considering student type, there was no 
significant difference in the overall frequency of tutoring use 
among the science, engineering and service students (see Tables 
1-3) . 

Older students reported more frequent use of tutoring than 
first year students. The utilization of the different tutoring 
options reflects and is consistent with the tutoring options 
available to the different student types and with what the 
students said during the Phase 1 interviews. 

Tf ynu nevsr or 1-2 times a quarter used tutoring 
KPrvices. whv did y mi not ao? In open-ended responses, 
students responded much as they did during the Phase 1 
interviews . 

Students preferred faculty tutoring if available. Student 
peer tutors were seen as being inexperienced and it was more 
uifficult to schedule a time with the peer tutor. 
Communication and general comfort level were better with 
faculty tutors. Most seemed to feel that groups were not an 
efficient use of their time. 

Some said they either do not have time for tutoring or 
don't need it. This was most common with first year and non- 
matriculated service students. Some selected comments follow: 

"... I have been under a lot of pressure with my 
school work... and seem to have a lack of time for 
tutoring. Usually, that caused me to lose my 
motivation to come for tutoring. Sometimes I was 
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behind in school work so that I wasn't ready to ask 
questions yet. " 

"I do not go for student tutoring. I do go for 
>- support faculty member tutoring because that person 

is available and my schedule matches with that 
i person's schedule easier than with the peer tutor's 

schedule. " 

-Group is slow. I prefer a tutor just for me. It is 
less time-consuming." 

-It's harder to think the same as other students who 
may be more intelligent than me or less. It's 
uncomfortable. . . " 

Ottastio j ?. how m uch, *n vou need tvtoring-taJafi-A 
mr ^ BB ful stu dent! Data from the questionnaire indicates 
that across all student groups the perceived need for tutoring 
with faculty (either individual or group) is greater than for 
peer tutoring (see Figure 2 and Tables 1-3). Older students 
report a greater need for tutoring than first year students. 
There was a tendency for service students to indicate less need 
for tutoring than that indicated by either science or 
engineering students. This is less so if they are older 
students. Across the board, peer tutoring is less preferred 
than faculty tutoring. This preference is less apparent for 
the engineering students but is consistent with the tutoring 
options available to them. Given the choice, students 
• generally indicate a strong preference for faculty tutoring. 
^ 11f fir s^n v how annri wjLfi the stor i ng you actually 
reived? As shown in Figure 1 and Tables- 1-3, the patterns of 
student responses are similar to those given for the questions 

o i r 
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about tutoring frequency and need. Students are generally well 
satisfied with tutoring services, but seem least satisfied with 
i. group tutoring with a peer, which is consistent with the 
£ options most available to them. There are no significant 
differences among the student types (science, engineering, 
service) in their satisfaction. Older students, however, are 
generally more satisfied than first year students. 

p 11 0 Efinn 4. H ™ wmild von like tO t,Ut.Pring Services 

proved? Many students emphasized that they prefer faculty 
tutors and suggested increasing the hours faculty are available 
to tutor, hiring more faculty tutors, and eliminating other 
faculty duties (specifically, teaching). Some suggested that 
group end peer tutoring times be more flexible and that each 
group session be held more than once. Those who depend most on 
peer tutors (mainly the engineering students and service 
students) suggested that peer tutors be good students, have 
experience with the course, and be better trained in 
communication and deaf awareness. One student suggested having 
more deaf peer tutors because of the extra benefit of having a 
deaf role model. A few suggested an open, walk- in, .no- 
appointment-needed, tutoring service for a limited time each 
day. Suggestions were made that sign-up schedules for all 
tutors be posted more in advance. 

Selected student comments are as follows: 

"I think peer tutoring would be helpful, especially 
by a hearing-impaired tutor so that students would 
see them as role models." 
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"Hire more peer tutors who are taking the same class 
with the 6ame teacher. Have more time open to 
students. Faculty should only schedule their other 
duties during students' class times and leave open 
the other times for tutoring students." 

"Improve the hearing student tutor's understanding of 
deafness. Some talk too much and don't realize we're 
not understanding... They shouldn't be discouraged 
with writing too much." 

"I feel that the services, although limited, are 
excellent. To improve services, faculty tutoring 
must expand . " 

n„ O R t - j^ 5. O ther student "omfflpntfl a^uf tutoring- 

Student responses to this open-ended question showed again 

that they were generally well satisfied with tutoring services 

and preferred faculty tutors. Yet, while preferring individual 

tutoring with faculty, they expressed a desire for increased 

networking, primarily for studying, with both hearing and 

hearing-impaired peers. The student comments also showed that 

expertise with the subject matter was not the only factor in 

determining a good tutor. Students felt a gooa tutor also 

needed to be able to communicate well, to have empathy, and to 

have good teaching skills. Some selected comments are: 

"The faculty tutors are very well educated and they 
usually know the materials presented in the class 
pretty well. Also, they have lots of sources 
available for students in their offices." 

"Well it's frustrating to not be able to meet with 
other' students to discuss subject matter. Worse, 
when the faculty member doesn't have enough time to 
spend with me . " 

" Like to have tutoring during weekends . 
Sometimes, I need help with my homework." 

"I feel I can learn a lot with group tutoring and 



group study sessions... I feel that a peer tutor with 
a group of students would be an ideal way to help 
develop better study and work habits . " 

"Tutoring by support faculty members is essential, 
especially in the first two years of college. It 
really makes a difference." 

"Tutoring was worthwhile. It taught me... things each 
time and double-checked my understanding to help 
build up my confidence." 

"In the past, I've had only peer tutoring, but now I 
have a faculty tutor. The peer tutoring wasn't as 
successful because the peer tutor did not have 
teaching skills." 

-...I'm more comfortable with faculty tutors because 
they know sign language and communicate well with 
deaf students. Also, they are more flexible with 
times and days. They give more time working with 
deaf students and I know they are motivated." 

Nntetakina 

During the Phase 1 interviews it became apparent that 
notetaking services were a major issue for the students. Many 
students said that their notes were incomplete, confusing, hard 
to read, or received too late to be helpful. 
Phase 2 

mipsl-ion 1 ■ How muc h rio vou nppfi notetaking? A year 
later, 85% of the questionnaire respondents said they need 
notetaking services "a let" or "very much" with the mean 
response falling between these two options (see Figure 1). The 
data in Tables 1-3 show that while science majors consistently 
reported a high need for notetaking regardless of year, 
engineering majors showed an increased need beyond the first 
year. 
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Question 2- How m u nh do vou need notetfrkjng? Unlike 
Phase 1 respondents, the majority of the students rated the 
j, quality of notetaking services as being very good, as the data 
jj in Figure 1 and Tables 1-3 indicate. 

The mean response to this question is 4.2 (between good 
and excellent) and when the students were grouped into 
respondent categories, there were no significant deviations 
from the mean and no significant differences among the groups. 

Q 1!PR tion 4. What, coul ri wp do to improve notet&kjnq 
RPrvices? To improve notes, many students suggested that all 
notetakers be trained before starting work and that a 
replacement always be obtained if the notetaker is absent. 
Students occasionally indicated that there is a need for notes 
to be available more promptly. Often it was suggested that the 
notetaker should have already taken the course. One student 
suggested that either the instructor or DSES faculty clarify 
and check the notes. Several students felt that more 
interaction between student and notetaker would help to improve 
the quality of the notes. 

Some respondents felt that the notetaker should write down 
as much as possible, almost creating verbatim transcripts of 
the class, while a few preferred the notetaker to condense the 
notes to only "important" information. 

Most students identified legibility, organization, good 
grammar and spelling, and completeness as characteristics of 
high quality notes and were the areas in which some notetakers 



9 

ERIC 



/ 



needed improvement. 

Comments indicated that English spelling, English grammar 
and style of handwriting may be causing some compounded 
difficulties when the notetaker is an international student. 

nnssHon 5. What sIbp would vou like to tell US &bOUt 
yr^ r nntatakinp p^riences or notetakinq needBl Satisfaction 
with notetaking appeared to be high. Students emphasized that 
the notes are extremely important to their success. There was 
some ambiguity as to the cnaracteristics of high quality notes 

Examples of student comments are as follows: 

"Most of the notes I have received were good and they 
helped me to be successful. In fact, I rely almost 
completely on the notes since I have little time to 
read the text . " 

". . .Notes were a very good backup source on the 
material ..." 

"I have been a successful student with the aid of 
notes . " 

"Throughout the first three quarters here at kit, 
I've learned to become an active user, using the 
notes as one of the best support services. I even 
sometimes take my own notes so I can compare with 
those of the notetaker." 

"I'd like to communicate more with my notetakers. If 
I have any questions about the notes, I sometimes 
can't communicate with the notetakers because they 
are too busy or don't know sign language." 

"Notes are the lifeblood of your educational needs." 

"Notetakers generally become weary when they have to 
put down lots of information. . . But they need to 
understand that we need as much as we can get." 

"The prompt service was great. But the notetaker 
didn't always take attendance correctly. Make 
attendance recording the responsibility of the 
student and not the notetaker." 



"Sometimes it's frustrating when the notetaker 
doesn't get a substitute. Also, the notetaker should 
have taker the course to know what's important." 

Advising 

Advising per se was not a topic mentioned often during the 
Phase 1 interviews, perhaps indicating it was not of major 
concern. One student pointed out that there was some 
resentment felt wnen asked about progress in certain courses. 
Phase 2 

Q yp.Bl-.ion 1. How often did vou aet this k.i nri of help tZW 
ESESJL A year later when students responded to the 
questionnaire, the most frequent kind of advising help students 
reported receiving was choosing courses and making career plans 
and registering for courses (see figure 1). Assistance in 
deciding on a major or decisions around job, co-op, and 
internship placement were used somewhat less frequently. 

When looking at respondents by major type, as in Tables 4 
and 5, many service students reported, as would be expected, 
never having received advising from DSES (even though service 
students reported a need for DSES advisement). Older service 
students reported more frequent use of advisement than did 
first-year service students. 

First year science and first-year engineering majors 
reported more frequent use of advising than the older students, 
specifically for choosing courses, help with registration, and 
deciding on a major. As would be expected, older students 
showed a higher frequency of advisement assistance for 
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placement. Regardless of year, engineering majors have more 
frequent advising contacts for choosing courses and deciding on 
'•■ a major than do science majors. 

QafifiUfiD 2 . nn W muc h flfi need this kind of help? AS 
can be seen in Tables 4 and 5, engineering majors indicate that 
their greatest advising needs are in the areas of choosing 
courses and deciding on a major. Science students report equal 
advising needs in all areas included in the questionnaire. 

pupation 3. how good has the advising vqu received? The 
data in Tables 4 and 5 indicates that students are generally 
satisfied with advising. 

^p S finn 4. what woul d impro ve adv i sing serv i ces ? 
Students generally felt that the advising services were good 
and that the encouragement and information they received were 
worthwhile. Some felt that advisors were not readily 
available. Occasionally, a student noted a particular 
unsatisfactory advising experience where either they did not 
agree with the advisor's viewpoint or felt the advisor was too 
hasty in expressing an opinion, did not have enough resources, 
or did not show adequate concern. 
Some specific comments are: 

"Be available more often. Each advisor is 
assigned too many students. I can't arrange my 
schedule to see my advisor." 

"The counselor is hard to find because she has 
two offices. So, it would be helpful if we had a 
full-time counselor." 

"Students should be free to select electives in 
their major. " 



"You could make efforts to relate to the student 
to see how the student is doing on a performance-wise 
basis . " 

"Improvement is needed to make sure students are 
on the right path in planning for courses each 
quarter. " 

"Advisors should have more experience advising 
and more options. . . " 

"Give more time for the students needs. Put up 
weekly schedule earlier. Set up 1-2 office hours 
when the advisor is free to see students right away 
if a problem can't wait until the next day." 

"You could hold forums getting students with the 
same major or just interested together to talk about 
their experiences and courses." 



pi pg-Hnn 5. what else wo uld vo v lito to te!3 us about 
advising? For the most part, students were pleased with 
advising services and like to have it available as a resource 
even though they prefer to be independent in their decision- 
making. Selected comments are: 

"Confidential information is to be kept 
confidential and I appreciate this a lot." 

"Helped me in many ways that I didn't realize. 
Couldn't have done better without this." 

"The advisors in DSES are well experienced in 
helping students decide on their majors, jobs, etc. 
They are really good advisors and they know what's 
good for the hearing-impaired." 

"Advisors do an excellent job... they are very 
helpful and extremely encouraging. I am really 
impressed with them for their patience and optimism." 

"I believe it is up to the student to decide how 
they should plan their schedule... There are some 
students that can make it in four years, so why have 
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st of them stay five years?" 



"Advising is very helpful, particularly how to 
start working on a co-op, deciding on courses, and 
where to get information...." 

General 

During the Phase 1 interviews, students reported that 
the heavy academic workload, the pace and the level of the 
academic work, created stress. Many students reported being 
caught between the deaf and hearing cultures with little 
interaction occurring between the two. Also, students 
mentioned struggling with the issue of independence versus 
dependence . 

For the Phase 2 survey, the above issues were put into 
questionnaire format. 
Phase 2 

riirnrir nfi ] ?. what mates it d i ff i m l f f a x you to be 
n c^P, B ful ptt Bt.udent? Students were asked to check as 
many of the seven issues that applied to or affected them. As 
Bhown in Figure 1, the most frequently mentioned were: a) The 
pressures of college life (71% of the respondents), b) the pace 
and level of the academic work (67% of the respondents), and c) 
the struggle to be independent while still needing services 
(55% of the respondents). The data in Table 6 indicates that 
the frequency of response to these issues was proportionately 
similar across majors and years of study. The Phase 2 
respondents reported few difficulties with the acceptance of 
their deafness, either on the part of other people or 
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themselves. When asked what additional problems make it hard 
' for them to be successful RIT students, many and varied 
i additional comments were provided. Some areas of concern are: 
Lack of accommodation and concern from RIT instructors, 
interfering family responsibilities, physical and emotional 
stress, lack of some basic learning skills, and lack of time 
management skills. For 25% of the respondents, social 
isolation and loneliness was a concern; when mentioned it was a 
major concern for that student. Selected comments in these 

areas of concern are: 

»I set too high standards for myself. Didn't do 
as well as expected based on my high school or NTiD 
grades . " 

"I try not to let social isolation bother me. 
But it feels better to be able to mix." 

"Social isolation from the hearing students in 
my classes is a problem and it's very difficult to 
cope with. " 

-The sudden transition from high school puts 
pressure on me to grow psychologically especially in 
the social area." 

"Organization, planning, study skills, 
communication, motivation, job finding, time 
management, health problems ... " 

"Falling behind. I'm always procrastinating. I 
don't have enough motivation to feel vitality. I 
needed better advice about how many credit hours to 
take and which courses." 

"I'm a foreign student and I particularly need 
support services." 

"The pace. Having to spend twice as much time 
as hearing students to undenstand . " 
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t rn fro, college to 1 interviews focused on 

being a student and the student's experiences with the DSES. 
Consequently, comments did not emerge about post-graduation 
anticipated difficulties. We included this item in the 
questionnaire since related concerns were presented by DSES 
members during preliminary discussions. 

The responses to the five options students could check 
indicated that all options were of some concern to this 
particular group of students. The need to communicate either 
informally in discussions with hearing peers or in written 
format is as important to them as their concern about finding a 
job in their major. For older engineering majors there is a 
greatly increased awareness of the importance of communication 
on the job. Additionally, older science students indicated 
that although communication and finding a job are important 
issues,, it will be hardest for them to leave the college 

environment behind. 

When asked what other problems they anticipated going from 
college to work, three significant areas emerged as important 
to the students. The students emphasized again communication 
issues of all types (writing, needing special devices, using 
the telephone, socializing, etc.). The second area emphasized 
was their acceptance by co-workers as an equal colleague. The 
third area was fear of the unknown and fear of failure. Some 
students mentioned that cooperative work experience has helped 
them to feel more confident and comfortable with their 
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anticipated transition to work. The following quotes 
illustrate this: 

M I've had 5 co-ops , each with a different 
company, so I wont have any problems. 11 

"Some people may be stereotyped about hearing- 
impaired people. I may be limited in some cases, 
mainly because talking on the telephone may be 
required for the job." 

"Being accepted because of my deafness, being 
informed of duties, cooperation with other workers, 
use of special equipment like FM unit, hearing aid 
and interpreter." 

"What to expect from a job. I mean I have 'fear 
of failure.' I'm not sure if I will succeed." 

Questions 5-8. Perbpective s on requesting DsSES services 

and on registration . During the Phase 1 interviews students 

mentioned the desire for a balance between flexibility in 

scheduling and a guarantee of services. The importance was 

noted for DSES to be clerically accurate in the designation o 

supported courses and the subsequent follow-up. 

Several students suggested the need to simplify the 

procedures for requesting support services. In general, 

students had a problem knowing about, and appreciating 

services. Older students strongly recommended that other 

students get services immediately in their first year, even 

though they themselves may not have done this. 

From these comments, four questions were written into the 

Phase 2 questionnaire as follows: 

Questio n 5, Do Y QU CQme tQ t he DSES support 
office before a new quarter fitfl r - fi to request support 
services? 



Seventy-eight percent of the respondents said they 

'always' go to the office before the start of a new quarter and 

request services and 22% either said they' sometimes' or 

'never' do. Table 8 breaks the 'sometimes' and 'never' 

responses into major type and level of student experience. It 

was apparent that the majority of 'sometimes' and 'never' 

responders were service students. 

Q aaStifiB 6 - Do tkfi pro cedures ws have now in 
science /math /enginee ring for requesting support 
f ifirvir.fis wor k all right for VOU? 

Ninety-three percent responded yes, that the current 
procedures work well for them. 

Qu estion 7. If y° u an swered no. what can we do 
to make it w ork better? 

Students suggested having more supported sections so that 
they are not limited to one particular time. Other comments 
indicated one student had a problem obtaining interpreters and 
another felt services had been reduced. One student used this 
opportunity to express a desire for the same interpreter 

throughout a course . 

Question fi. Is the system that limits VOU to 
registering only fo r supported sections seriously 
affecting your s uccess as a student? 

Sixty-two percent of the students who responded said, 

"no," the current registration system is not seriously 

affecting their success as a student and 38% said ■yes" it is. 

As shown in Table 9, those believing it does affect them are 

primarily older students. 



When asked why they felt this way, students indicated that 

the limitations caused schedule conflicts and resulted in 

inconvenient schedules. Generally, however, the problems were 

eolved if they sought assistance from DSES. The most 

significant problem related to limited choice of courses was 

that students could not always have the teacher of their 

choice. Students emphasized that the support services are 

crucial to their success, so if they must take a course that is 

not supported it is a serious problem. This appears to happen 

very seldom. Representative comments follow: 

"Causes too many conflicts. Conflicts cause me 
worry. Convenient times are not usually supported." 

"So far I've always gotten support, even if I've 
signed up for non-supported sections." 

"Because I can learn more when I choose a class, 
since I may know the teacher or students taking the 
class . " 

"Not a big problem but it makes it hard to 
choose courses to fit well into my schedule. Advisor 
usually can solve the problem. " 

"...Would like to have college math for NTID 
only, like NTID sections of Liberal Arts." 

"I registered for a supported section and all of 
the other deaf students were in another undesignated 
section. " 

"No interpreter = no comprende; 
one support faculty member «= near zero time; 
ineffective student tutor = frustration; 
3 strikes ■ I'm out! " 

ft^opi-ir m 9. fij-nflents' perspectives of the important 
nB pftr.t.B of f h » g^o rt * a Hl1tY mfiffifcPJLlB ^ ob - In the phase 1 
interviews the support faculty role was not clear to students. 
The students' responses to this open-ended question were all 



very individualized, and seemed to reflect what that student 
needed and desired from their support faculty member. Many 
students mentioned tutoring and the emotional support, 
understanding and encouragement that they received from the 
support faculty as being important aspects of the faculty's 
role. Selected comments are: 

"...Understanding deafness and how to communicate 
with the hearing-impaired." 

"...To discuss problems that need clarification and 
pinpoint errors; have group tutoring to promote 
interest and learn from other peers." 

"Understanding the students' needs and then helping 
them acquire the proper help." 

"Encouraging students to learn on their own, adjust 
to the pace and to learn to use study skills 
effectively . " 

"The ability to relate educationally and personally." 

"To understand students' needs and weaknesses so that 
they can better help students to understand concepts; 
spend a lot of time if needed." 

Q n&stian iq. what stu d ents sav thev do in tte classroom. 

This question and its response options were directly extracted 
from comments provided by RIT faculty during the Phase 1 
interviews, students responding to the Phase 2 questionnaire 
indicate that 88% see instructors outside of class for help and 
66% ask questions in class but only 52% participate in class 
discussions and 46% take their own notes. Students assume more 
responsibility in these areas across all majors as they mature 
(see Table 10). Open-ended responses related to other active 
learning behaviors that students use and reasons for non-use of 
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those listed generally focused on two major factors: a) <x 
desire to work with and learn from hearing and hearing-impaired 
peers in a student network and b) a lack of confidence in 
participating in front of hearing-impaired students and in 
approaching the classroom instructor. Comments illustrative of 

these concerns are: 

"I have to admit, I don't ask questions for I am shy 
and afraid to ask dumb questions. I don't work with 
the hearing students, either since I feel a wall 
between me and them and I figured they probably felt 
the same about me." 

"I work with other students in study groups." 

"I try to ask other students questions and to get to 
know them personally. Also, I have had tremendous 
support from the instructor when asking for help." 

O ^pHnn 11. what wou ld help vou to take a more active 

role in the classroom? Responses showed that 55% of the 

students valued assertiveness , 38% public speaking training, 

and 41% informal social events with instructors as aids to 

increasing their confidence and interaction. The perceived 

importance of these activities was generally the same across 

all student groups. Of the 32% indicating a desire for 

training in taking their own notes, the science and service 

students showed the most interest (see Table 11). Other 

assistance students would like to have in order to increase 

their skills as active learners was: 

1. Social activities with hearing students and with 
tutors to increase interaction. 

2. Group study sessions with hearing students. 
3*. Efforts to increase hearing students and 
faculty's awareness of deafness. 

4. Training in formulating questions, using correct 
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grammar, developing good writing skills and 
effectively using the textbook. 

Question 12. Which DSES courses have VQU ta ken? 

DSES faculty noted the lack of attendance in their courses and 

questioned whether students appreciated the importance of the 

courses. It was apparent from the Phase 1 interviews that 

students were uninformed about many services provided by DSES, 

including those courses. This question was included in the 

questionnaire asking students about the DSES course offerings 

as a means of informing them of the existence of the courses. 

The most frequently taken courses were DOCL (5 5% of the 

respondents) and Learning Strategies (25%) followed by Reading 

and Thinking in Science and Technology (21% of the 

respondents ) . 

Qups-Hon 13. Feeling f? ft bout the DSES course taken. 
Comments showed that students had mixed reactions about the 
DSES courses they had taken. Many found them helpful and 
valuable. Other students resented having to take them. 
However, as students matured and looked back, they realized the 
importance of the courses to their studies. Selected comments 
are : 

"Too easy and not worth taking DOCL... A waste of 
time because student can ask people the questions 
quickly about where to go or about what can be 
offered (from a student who took 3 DSES courses]." 

-It really helps because there are a lot of little 
things that I missed and now it helps me a lot (from 
a student who took 2 DSES courses]." 
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"I felt that they were helpful a£Le_r I finished 
taking them. I learned much better after taking them 
than I had before them [took 3 DSES courses]." 

■Learning Strategies helped me realize how much more 
there is to, studying and preparation. I felt the 
Reading and Thinking helped me understand the need to 
read instructions carefully [took 2 courses]." 



pupations 14 and is. Participation in the Big Brother /Big 
f?i Pt er Program . The Big Brother/Big Sister Program was 
recently implemented (Fall, 1986) and only 16% of the 
respondents said that they had participated in it. When asked 
about the value of the program in Question 15, more students 
than those who had actually participated in the program 
responded. Therefore, respondents unfamiliar with the program 
still gave it a rating. With this skewed response in mind, 34% 
of the 56 students responding to the questionnaire said that it 
was valuable. 

Q aastifliifi 16 and 17. particip a tion i n D SES soc i a l events. 
In the Phase 1 interviews DSES faculty questioned the value of 
continuing social events for students since attendance at these 
events had been low. The data from the questionnaire showed 
that the majority of students who said they did not attend 
these events were first year and service students (see Table 
11). When asked to rato the importance of these events, older 
students rated them as being more important than did first year 
students (see Table 12). 

ftup.Btion 18 how co nid dses social events be improved? 
When asked how social events could be improved students 



* 3'J 
ERIC 



suggested scheduling events to be held earlier in the quarter, 
involving students more in implementation and planning, 
avoiding time conflicts, providing regularly scheduled events 
at the same times each quarter, and informing students earlier 
of scheduled events. 

Question 19. Advice for new cross-registered students in 
ESES^ The advice provided by respondents for new students 
focused on the importance of support services and or. balancing 
the desire for independence with a healthy use of assistance. 
The following quotes illustrate the force with which the 
students hold these beliefs. 

"Just not to be too dependent, but also keep in mind 
that it is good for you to use their services." 

"Get all the support you can get because it really 
helps you to compete with hearing students in your 
class and helps you to become successful." 

"Meet as many faculty members and students as you can 
at the beginning of the quarter." 

"Study hard. " 

"Make sure that you know your basics and be ready to 
take on a challenge. You may have to give yourself a 
lot of time to be successful at RIT, but look at the 
whole!" 

"Don't set your standards too high at first. Just do 
the best you can and when down, don't give up." 

"Have a fighting spirit and do not give up easily. 
Work hard and the rewards will be great... Have a 
positive attitude all the time and take advantage of 

the outstanding tutoring services offered by DSES. 
I praise their great tutoring sessions!!!" 

"Take advantage of the support. You control your own 
destiny with support rather than the other way 
around . " 



"We, the students, should set up a meeting for new 
cross-registered students so that they can ask any 
questions before they face new experiences in RIT. " 

"Work hard; be aggressive in your learning..., get 
what you want to know. Life here is not the same as 
NTID. M 

"Be responsible for your own education. The faculty 
and staff are resources to use. Be assertive and ask 
even if you think it's dumb. Create a personal 
relationship with instructors and support faculty 
through tutoring so they know you are going beyond 
what is required. Keep you sights set on your long- 
term goal and balance frustrations and 
disappointments with reality." 

"Start tutoring earlier!!! Don't wait a until a week 
before an exam and learn time-management skills." 

Question 20. Additional su g gestions and comments. 

Students were asked to provide comments and perspectives on 

aspect of DSES that would strengthen the services offered. 

Selected comments are: 

"There is not much to improve, but DSES has helped me 
a lot and without them I could not be still going 
here . " 

"I feel DSES is doing a good job trying to meet the 
needs of the students every way they can and the 
services should not be cut in any way pleaSfi! ! 
Without them, I would not have notetakers for my 
classes and God help me if I don't do well. 

"I've heard that some students feel that some staff 
members have cold shoulders. Some staff just tutor 
and do not care about what's happening. Perhaps it 
would be helpful to have staff members relate to the 
students more and show concern. Sometimes it's nice 
to know the staff instead of knowing them only as 
tutors. Also, it's sometimes helpful if the tutor 
teaches the same material as was taught in lectures. 
Otherwise, confusions are created." 

"Keep up the good work. I feel at home in the 
Science support office more than any place on campus, 
including my own support team office. There is a 
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tangible spirit of cooperation and respect. The 
friendliness, availability of faculty and the 
conference room with library are really pluses in my 
opinion. " 

"Get more hearing students involved in helping 
promote ways to increase better understanding of our 
academic differences and have them help NTID students 
in studying with group sessions for working on 
homework... I tend to feel, this would help us both 
become used to how to deal with our differences in 
how we communicate. Thus, it would lead to 
friendships and better working relationships for 
future generations." 

Summary and Con clusions 
Contrary to the Phase 1 interviews which showed a possible 
problem with notetaking services, students were generally well 
satisfied with all support services. Quality of services was 
not an issue with the students. Instead, the problem appeared 
to be that students could not get enough of what they 
considered a good thing. The reasons for this appeared to be 
that students have limited time flexibility and therefore can 
not always obtain assistance at the times when it is made 
available. And also that support faculty are often fully 
occupied. For some students, the traditional methods of 
delivery of services did not suit their particular needs. 
There appeared to be as many individual needs as there were 
students and to satisfy all students an unlimited number of 
options would be necessary. 

Students continue to struggle with a need for independence 
while realizing a concomitant need for services. Students at 
different maturity levels appeared to value and use services 
differently. Experienced students often realized that they 
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should have used services differently in their earlier years. 
This was reflected in their advice to new students. It was 
encouraging also to note that growth and positive attitude 
changes were occurring as the students advanced in their 
studies and many students attributed these changes and their 

success in part to DSES . 

Some problems students had with support service appeared 
to relate not to the quality of services but to the frustration 
inherent in having to structure their learning in a way 
different from that of their hearing classmates. 

Students preferred individual tutoring with a support 
faculty member over the other options listed in the 
questionnaire. This was consistent with Phase 1. At the same 
time, however, they wanted the feedback they thought they could 
obtain by working with their hearing and deaf peers. 
Respondents showed an awareness that there is an attitude 
component to tutoring and that a good tutor not only provides 
information but is emotionally supportive as well. 

Notetaking, contrary to Phase 1, was also viewed as 
providing quality services. Students did suggest that 
notetakers be trained prior to starting work and that delivery 

of notes be prompt. 

There appeared to be no one definition of good notes. No 
one notetaking style or format will satisfy all students. For 
example, some students wanted a verbatim transcript while 
others wanted an outline of only salient points. 
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The results of this evaluation regarding notetaking were 
in agreement with a report from the Tutor Notetaker Advisory 
Group (June 1988) in which it was stated that students rated 
the quality of notes as good. Other research done at NTID on 
support services was considered irrelevant for comparison, the 
reason being either differences in the population or design of 
the study (For example - Stinson, Saur & Panara 1982). 

Pressure of college life and the fast pace of academic 
work were judged as important concerns by students. For some 
students, social isolation was a serious concern. 

Communication, both oral and written, loomed as 
potentially a problem for students as they make the transition 
from college to work. 

Students show an awareness that they need to improve not 
only their communication skills but also their basic skills, 
study habits and time management. This may indicate that the 
DSES courses are having an impact. 

Students do not categorize advising as a separate service 
but see it as incorporated in all of their contacts with 
support faculty and they define the support faculty role very 
broadly to include many advising components. 

From the student's perspective assertiveness training 
appeared to be a significant need. 

Responses to open ended questions throughout the 
questionnaire often included suggestions which indicated a 



desire to work with and learn from their peers. Strong support 
of the Big Brother/Big Sister program is an example. 

Students valued the social events held by DSES but often 
could not fit them into their schedules. They felt that if the 
events were scheduled on a regular basis they would be able to 
plan ahead and attend. 

The respondents valued the support services DSES provides 
and define the role of a support faculty member very broadly. 
Students strongly emphasized that the warmth, acceptance and 
empathy which they see as best provided by a support faculty 
member is important to them. 

Recommendations 
1. It was recommended in Phase 1 that opportunities for 
students to interact with peers be increased. One move in that 
direction which has already been accomplished is the Big 
Brother/Big Sister program. A further recommendation would be 
to continue to strengthen the Big Brother/Big Sister program. 

The opening of the engineering study room after 5 p.m. 
also has helped to increase student interactions and responded 
to a recommendation in Phase 1. 

Additional opportunities for students to work together 
with both their deaf and hearing peers should be developed. 
This might include among other possibilities establishing study 

groups, using more deaf peer tutors and providing sign language 
training for hearing students. 



Some of these peer group interactions could be designed 
to serve as a preparatory environment for communication and 
interacting on the job. 

Potential NTID transfer students might particularly 
benefit from early involvement in these peer interactions. 

2. Assertiveness training could be made more readily 
available to students. 

3. Continue to provide quality support services to the extent 
that they are now being provided. Look also for opportunities 
to be creative in making available innovative options to meet 
the needs of students with a variety of learning styles and who 
have limited time flexibility. 

4. Make students better aware of the many and varied services 
offered by DSES . Acquaint them with the constraints within 
which DSES must operate so that their expectations do not 
become unrealistic and with the philosophy of support services 
so that they may become more comfortable with their need for 
services and more discriminating in their use of support 
services . 

5. Continue to solicit student input when considering program 
changes or modifications. For instance, when students were 
approached about this evaluation and had the opportunity to 
have their opinions heard and responded to by DSES, they were 
very open and responsive and gave much thought to their input. 

A research approach in which the salient factors are 
determined by the affected population and emerge from data 



provided by them, rather than the more usual approach of 
imposing a predetermined structure on the data collection was 
very useful in identifying areas of importance and concern to 
students . 

Consider the value of this methodology if another program 
evaluation is done by DSES and if time will permit this more 
lengthy approach. 

6. Considering that students often value support services and 
other offerings of DSES more as they mature, drive ways to 
impress upon newer and younger students the value of these 
services and offerings and to increase their comfort in taking 
advantage of both academic and social DSES programs. 

7. Students define the role of the DSES support faculty 
member as one in which the students' needs are paramount. 
Actively seek ways to make available more time for student 
contact and to reward student centeredness without fostering 
undue student dependency. 

8. Reaffirm a commitment to the maturing nature of an early 
cooperative work experience and the value this has for 
instructing students in job expected behaviors and in the need 
to improve their communication, interpersonal and learning 
skills . 

9. Discuss as a team the department's identity and mission. 
For example, is the goal for students one of assimilation or 
one focused on the deaf culture? Once the department's 
identity and mission are better understood, options related to 



operations should become clearer, decisions should be less 
complex and consensus perhaps easier to obtain. 
10. Not all students felt socially isolated at R.I. p. But 
those who did, it was for them a particularly disturbing 
problem. Ways should be explored to assist this group of 
students to deal with their need to assimilate through 
counseling and other activities. 



DEPARTMENT OF SCIENCE AND ENGINEERING SUPPORT (DSES) 

STUDENT QUESTIONNAIRE 



Year in cross-registered programs 
Major 



Number of scien,e/math/engineering courses you are taking this quarter * c 1. \, «<w< _ c - 



TUTORING 

The Department of Science 
are: 



and Engineering Support (DSES) offers four kinds of tutoring. These 



A Individual tutoring with a support faculty member 

D. GrSup tutoring (2 or more students) with a student (peer) tutor 
Please think about your experience in science/math/engineering and answer the following 
questions. Circle the answer that best shows how you feel. 

, How many, ti mPS do you usually no for tutoring in science/math/engineering 



courses? 



MOHfc 
THAN 



1.1T1MIS >-5TlMI5 ONCf ON« 

NEVER AQUAiTfi A OUAHTCi AVWU 

ift 2(^) 3<V) «4(u^ 5^J 

2(uVa6^ 4 to 5<te 

iW 3(>) 4M 5(2) 

If you circled (1) never or (2) 1-2 times a quarter, why did you no, go for this kind of tutoring? 



il/S A. individual tutoring with a support faculty 
member 

. : ^ B. Individual tutoring with a student (peer) tutor 

■5 j S C Group tutoring (2 or more students) with a 
support faculty member 

v ( D Group tutoring (2 or more students) with a 
" v ' student (peer) tutor- 



2. 4 How much do you need tutoring to be a successful student? 
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Individual tutoring with i 
member 

Individual tutoring with 

Group tutoring (2 or moi 
support faculty member 

Group tutoring (2 or moi 
student (peer) tutor 



3. How good was the tutoring you actually received? 



Individual tutoring with a support faculty 
member 

\ y \ B. Individual tutoring with a student (peer) tutor 

Group tutoring (2 or more students) with a 
support faculty member 

Group tutoring (2 or more students) with a 
student (peer) tutor 





.^l A. 




: 1, \ B. 
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NOT AT A SOME- A VERY 

ALL LITTLE WHAT LOT MUCH 
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POOR FAIR 



OK 
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4. What could we do to improve tutoring services? 



5. 



What else would you like to tell us about your tutoring experiences or tutoring needs? 



NOTETAKING 

Please think about your experience in science/math/engineering and answer the following 
questions. 

1 . How much do you need notetaking to be a successful student? 

NOT AT ALL A LITTLE SOMEWHAT ALOT VERY MUCH . 
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2. How good are the notetaking services you usually receive? 

POOR FAIR OK GOOD EXCELLENT 

x - %x 1 fa 2 3 W 4 5 M 

3. What could we do to improve notetaking services? 



4. What else would you like to tell us about your notetaking experiences or notetaking 
needs? 



ADVISING 

The Department of Science and Engineering Support (OSES) offers the following help to 
students. 

A. Help choosing courses and making career plans 

B. Help deciding on a major and changing or applying to a major 

C Help with registering tor courses (add/drop, pre-registration, etc.) 

D. Help finding counseling when you need it 

E. Help with job, coop and internship placement 

Please think about your experience in science/math/engineering and answer the following 
questions. 

1 . How often did you get this kind of help from DSES? 









MORE 








1 NAN 

2 OR 3 EVERY once a 
NEVER ONCE TIMES QUARTER , quarter 
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Help choosing courses and making career plans 
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-O^ B. 


Help deciding on a major and changing or 
applying to a major 










Help with registering for courses (add/drop, 
pre-registration, etc!) 
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Help finding counseling when you need it 
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Help with job, coop and internship placement 
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2. How much do you need this kind of help to be a successful student? 



0'JW\-it\9S.J NOT AT A SOME- A VERY 

ALL UTTLE WHAT LOT MUCH 

r^j.^A. Help choosing courses and making career plans 2 ^ 3(\ f £) 4 (v^ 5(10 

O.^B. Help deciding on a major and changing or - A ^ 2 / 0 "S if/A 4 AA 5 ft 

1 applying to a major ' v ^ v ^ ; ^ v JV ' 

a 1 C. Help with registering for courses (add/drop, 1 2 3*!^ 4 fa) Sr) 

pre-registration, etO r ' 

: i),0 D. Help finding counseling when you need it 1 

O.^ E. Helpwith job, coop and internship placement 1 (W) 2 (V) J3 6°) 4^*) 5 ft) 



I. How good wa s the help you actually received from OSES? 

( flo^vli x - ■ EXCEL- 

POOR FAIR OK GOOD LENT 

-0.1 A. Help choosing courses and making career plans 1 (C) 2 (V) 3 fal j 4 (rs) S(%) 

-VI B. Help deciding on a major and changing or 1 /A 2 IJ) ,/,/ 4 /^) 5 /£ 

applying to a major K ' 1 v \ 

:^^C. Help with registering for courses (add/drop, 1 l^) 2 (1) 3(V<) V 46 s / 5^) 

pre-registration, etc. J 

- "i-^D. Help finding counseling when you need it 1 (*>) 2 (m) 3 (W) 40*) 5/"', 

•s'h 'b E. Help with job, coop anu internship placement , ^ 2 3(1") 4 6?-) 



4. What could we do to improve our advising services? 



5. What else would you like to tell us about your advising experiences or your advising 
needs? 
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GENERAL 

1 Do any of the problems listed below make it hard for you, personally to be a successful RIT 
student? Please put a check in front of all that you believe apply to you. ( r^v q ^»' • ••--) 
the struggle to be independent while still needing services 
language skills 
fa} your acceptance of your deafness 
fcg) the pressures of college life 
(■7jj\ the pace and the level of the academic work 
(\\.\\ social isolation (being lonely) 
(\^ other people's acceptance of your deafness 

2. What other problems are there that make it hard for you to be a successful RIT student? 



3. What will be hard for you when you go from college to work? Please check all that you 
believe apply to you. «\j«fic\ «•&) 

Aq\ leaving the deaf college culture behind 
(\c\) finding a job in your major 

( a ^) needing to communicate with hearing people on the job 
( \ M ) being independent (on your own) 
(\e\^ writing reports, letters, etc. on the job 

4. What other problems do you think you might have going from college to work? 



Do you come to the DSES support office before a new quarter starts to request support 
services? Check your answer. LK-<i s *v«. « v o) 

never (V) sometimes (^-z) always x - P-l 

Do the procedures we have now in science/math/engineering for requesting support 
services work all right for you? i«.»v^ 
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If you answered no, what can we do to make it work better? 



Is the system that limits you to registering only for supported sections seriously affecting 
your success as a student? ( T^^6>w^n<iv^) 



0 No [I] Yes If yes, why?' 



What do you think are the most important parts of the support faculty member's job? 



RIT faculty feel that tne n i iu» students should take a more active part in the cLss.com 
and work more directly with their instructor and hearing peers. Which of the following 
do you regularly do? Check all that apply to you. (vrt.?^?!--: 



take own notes 



(yyC^ see instructor outside of class for help 
f participate in class discussion 

ask questions in class 
(\^) other -what? 



none of the above - Why not? 



4C 



1 1 . What would help you to do more of the things listed in question 1 0? Check all that apply 
to you. C r^c ^cwc \ 

I \ ^ training in how to take your own notes 

(zA training in publicspeaking 

( h\) assertiveness training to gain confidence 




informal social events to meet with your instructors 
other - please be specific 



12. Please check the courses offered by the Department of Science & Engineering Support that 
you have taken. (r^Q^^^c^ 

(^\) Dimensions of College Life (DOCL) 

( \v\j Learning Strategies (Study Skills) 

Reading & Thinking in Science (Thinking Skills) 

f ^) Career & Life Planning 

(%) Coop Preview 

( Q Colloquia 

13. Please tell us yourfeelings about the OSES courses ysu have taken. Please be specific. 



14. Did you participate in the DSES Big Brother/Big Sister program? 

Yes, I was a 

(hH) No 

1 5. How valuable is this program? 

(V <lr- Okj t7vic-i 

NOT AT ALL ' 

VALUABLE A LITTLE SOMEWHAT A LOT VERY VALUABLE 

_ 1 2 3 V y 4 5. 

^ <J) (ti A (j) ((*) 

1 6. DSES sponsors social events such as picnics, winter party, open "house" for Halloween and 
Christmas, volleyball, etc. Did you go to any of these events this past year? 



ho) yes h^j no 
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BEST COPY AVAILABLE 



17. How important to you are these events? 

!m°poVtan L t auttie somewhat a lot very 

1 2 3.4 5 

18. How could' we improve these events? 



1 9. Do you have any advice for new cross-registered students in DSES? 



20. Please give us your comments or suggestions to help us improve the work done by DSES, 



r r 

U 'J 



Binder.G.-002## 
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TABLE 1 

MEAN RESPONSES TO TUTORING AND NOTETAKING 
ITEMS FOR FIRST-YEAR STUDENTS 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


ccDv/irc 


I. A. Freq. of indiv. with fac. 


3.2 


4.0 


2.7 


B. Freq. of indiv. with peer 


1.0 


1.9 


1.5 


C. Freq. of group with fac. 


2.8 


2.4 


1.8 


D. Freq. of group with peer 


1.0 


1.3 


1.0 










II. A. Need for indiv. with fac. 


3.6 


4.1 


2.7 


B. Need for indiv. with peer 


1.8 


2.7 


1.8 


C. Need for group with fac. 


3.3 


3.0 


2.0 


D. Need for group with peer 


2.0 


2.3 


1.2 










ill. A. Satis, with indiv., with fac. 


4.4 


4.1 


3.6 


B. Satis, with indiv., with peer 


2.0 


2.9 


3.0 


C. Satis, with group, with fac. 


3.6 


3.7 


2.8 


D. Satis, with group, with peer 


2.0 


2.5 


1.0 










IV. A. How much do you need 
notetaking? 


4.8 


4.0 


0 0 


B. How good are v.he notes? 


4.5 


3.9 


4.2 


n = 


6 


7 


6 



TABLE 2 

MEAN RESPONSES TO TUTORING AND NOTETAKING 
ITEMS FOR OLDER STUDENTS 



ITEM 


SCIENCE 

MAIOR 
IVlMJ \Jt\ 


ENGINEERING 

MAIOR 


SERVICE 


1. A. Freq. of indiv. with fac. 


A C 

4.5 


3.5 


A 0 

4.3 


B. Freq. of indiv. with peer 


2.8 


^ A 

3.4 


2.2 


C Freq. of group with fac. 


3.8 


"> A 

2.4 


3.1 


D. Freq. of group with peer 


1 . 1 


2.4 


1 Q 
1 .O 










II. A. Need for indiv. with fac. 


4.1 


A 0 

4.3 


*3 Q 

3.y 


B. Need for indiv. with peer 


z.y 


3.0 


Z.J 


C. Need for group with fac. 


O *7 

3. 1 


3.U 




D. Need for group with peer 


Q 

Z.O 


i. 

2 b 


— r, — 

1./ 










III. A. Satis, with indiv. , with fac. 


4.0 


A A 

4.4 


4.D 


B. Satis, with indiv., with peer 


3.3 


3.5 


2.9 


C Satis, with group, with fac. 


4.1 


3.5 


4.4 


D. Satis, with group, with peer 


3.0 


2.5 


2.6 










IV. A. How much do you need 
notetaking? 


4.7 


4.8 


4.1 


B. How good are the notes? 


4.4 


4.0 


4.2 


n = 


10 


13 


10 



(J 
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TABLE 3 

RESULTS OF ANOVA FOR TUTORING AND 
NOTETAKING ITEMS BY MAJOR 
FOR ALL STUDENTS 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE 


1. A. Freq. of indiv. with fac. 


4.07 


3.76 


3.63 


B. Freq. of indiv. with peer* 


2.14 


2.95 


1.79 


C. Freq. of group with fac. 


3.43 


2.52 


2.58 


D. Freq. of group with peer* 


1.07 


2.14 


1.42 










II. A. Need for indiv. with fac. 


4.08 


4.21 


3.56 


B. Need for indiv. with peer* 


2.58 


3.21 


2.06 


C. Need for group with fac. 


3.50 


3.05 


2.63 


D. Need for group with peer** 


2.25 


2.63 


1 .44 










III. A. Satis, with indiv., with fac. 


4.40 


4.22 


3.57 


B. Satis, with indiv., with peer 


3.60 


3.28 


2.57 


C. Satis, with group, with fac. 


3.40 


3.61 


3.43 


D. Satis, with group, with peer 


2.80 


2.61 


2.14 










IV. A. How much do you need 
notetaking? 


4.75 


4.52 


4.16 


B. How good are the notes? 


4.44 


3.95 


4.21 


n - 


16 


20 


16 



^Significant differences on this item exist at p < .05. 
^Significant differences on this item exist at p < .01. 



TABLE 4 

MEAN RESPONSES TO ADVISING ITEMS FOR 
FIRST-YEAR STUDENTS 



ITEM 


SCIENCE 

ft. ft A 1 n 

MAJOR 


ENGINEERING 1 
MAJOR 


SERVICE 


1 ST YEAR STUDENTS 








j. m. rrtrq. wi in tnuuiiny cuurici 


3 7 

j. / 


3 Q 

J.J 


1 ft 


B. Freq. with deciding on a 
major 


2.8 


3.2 


1.0 


C. Freq. with registering 


3.5 


3.3 


1.2 


D. Freq with counseling 


3.6 


3.2 


1.7 


E. Freq. with placement 


2.0 


1.8 


1.0 










II. A. Need for choosing courses 


3.5 


3.7 


2.0 


B. Need for deciding on a major 


3.2 


3.5 


1.7 


C. Need for registering 


3.5 


2.6 


2.0 


D. Need for counseling 


3.4 


2.9 


2.0 


E. Need for placement 


2.6 


2.8 


1.8 
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B Satis with dpridinn on a 
major 


3.2 


4.2 


2.5 


C. Satis, with registering 


3.5 


4.0 


2.6 


D. Satis, with counseling 


3.3 


3.8 


2.8 


E. Satis, with co-op placement 


3.5 


3.0 


2.3 


n = 


6 


7 


6 



TABLE 5 

MEAN RESPONSES TO ADVISING ITEMS FOR 
OLDER STUDENTS 



ITEM 

Ilk l¥l 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE 


OLDER STUDENTS 








1. A. Frtfq. with choosing courses 


3.5 


3.7 


2.3 


B. Freq. with deciding on a 
major 


2.4 


2.8 


1.6 


C. Freq. with registering 


3.3 


3.3 


3.1 


D Freq. with counseling 


3.0 


3.4 


2.1 


E. Freq. with placement 


2.4 


3.0 


1.4 










II. A. Need for choosing courses 


3.4 


4.2 


2.8 


B. Need for deciding on a major 


3.3 


3.1 


2.4 


C. Need for registering 


3.2 


3.7 


3.0 


D. Need for counseling 


3.2 


3.9 


2.5 


E. Need for placement 


3.7 


3.6 


1.8 










III. A. Satis, with choosing courses 


4. 1 


4.1 


3.1 


d. jails, wiin ueciuing on a 
major 


3.0 






C. Satis, with registering 


4.3 


4.1 


3.8 


D. Satis, with counseling 


3.5 


4.0 


3.1 


E. Satis, with co-op placement 


3.3 


3.8 


2.5 


n = 


10 


13 


10 



TABLE 6 

FREQUENCY OF RESPONSES TO OPTIONS IN QUESTION: 
"WHAT MAKES IT HARD TO BE A SUCCESSFUL STUDENT?" 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE 


1ST YEAR STUDENTS 








struggle to be i ndependent 


3 


2 


2 


language skills 


2 


2 


3 


acceptance of deafness 


2 


0 


2 


pressures of college 


4 


6 


2 


pace and level of work 


3 


5 


3 


social isolation 


0 


3 


1 


others' acceptance of deafness 


2 


0 


1 


n = 


6 


7 


6 










OLDER STUDENTS 








struggle to be independent 


5 


8 


7 


language skills 


4 


6 


1 


acceptance of deafness 


0 


1 


1 


pressures of college 


7 


10 


8 


pace and level of work 


6 


10 


7 


social isolation 


6 


2 


1 


others' acceptance of deafness 


2 


2 


2 


n = 


10 


13 


10 
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TABLE 7 

FREQUENCY OF RESPONSES TO OPTIONS IN QUESTION: 
"WHAT WILL BE HARD GOING FROM COLLEGE TO WORK?" 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE I 


1ST YEAR STUDENTS 








leaving college behind 


3 


2 


1 


finding a job 


2 


2 


2 


needing to communicate 


3 


0 


1 


being independent 


2 


1 


4 


writing 


4 


2 


2 


n = 


6 


7 


6 










OLDER STUDENTS 








leaving college behind 


5 


2 


3 


finding a job 


4 


5 


3 


needing to communicate 


4 


9 


3 


being independent 


1 


0 


4 


writing 


1 


8 


3 


n = 


10 


13 


10 
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TABLE 8 



NUMBER OF STUDENTS WHO SAID THEY SOMETIMES OR NEVER 
GO TO THE OSES OFFICE BEFORE A NEW QUARTER STARTS 



ITEM 


MAJOR 


trsjoirJttKiNu 
MAJOR 


I 

SERVICE 


1ST YEAR STUDENTS 








Sometimes 


0 


0 


3 


Never 


1 


0 


1 


n = 


6 


7 


6 










OLDER STUDENTS 








Sometimes 


1 


2 


2 


Never 


0 


0 


0 


n = 


10 


13 


10 
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TABLE 9 



NUMBER OF STUDENTS WHO BELIEVE THE REGISTRATION 
SYSTEM SERIOUSLY AFFECTS THEIR SUCCESS 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE 


1ST YEAR STUDENTS 


3 


2 


3 


n = 


6 


7 


6 


OLDER STUDENTS 


3 


5 


4 


n = 


10 


13 


10 



5 



TABLE 10 

WHAT STUDENTS SAY THEY DO IN THE CLASSROOM 

(FREQUENCY OF CHECKS) 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE 


1ST YEAR STUDENTS 








take notes 


1 


5 


5 


see instructor 


4 


6 


6 


participate in class 


1 


4 


6 


ask questions 


5 


4 


4 


other 


1 


0 


0 


none 


1 


0 


0 


n = 


6 


7 


6 










OLDER STUDENTS 








take notes 


6 


4 


5 


see instructor 


9 


13 


8 


participate in class 


4 


6 


6 


ask questions 


8 


9 


6 


other 


5 


5 


3 


none 


0 


0 


1 


n = 


10 


13 


10 
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TABLE 11 

WHAT WOULD HELP YOU TO DO MORE IN THE CLASSROOM? 

(FREQUENCY OF CHECKS) 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE 


1ST YEAR STUDENTS 








training in notetaking 


4 


1 


2 


training in public speaking 


1 


4 


2 


assertiveness training 


4 


4 


3 


informal meet with instructors 


2 


2 


4 


other 


2 


0 


0 


n = 


6 


7 


6 










OLDER STUDENTS 








training in notetaking 


4 


1 


4 


training in public speaking 


4 


5 


4 


assertiveness training 


8 


8 


2 


informal meet with instructors 


3 


6 


4 


other 

•« 


3 


0 


1 


n = 


10 


13 


10 
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TABLE 12 

DID YOU GO TO ANY DSES SOCIAL EVENTS? 
(FREQUENCY OF CHECKS) 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE 


1ST YEAR STUDENTS 








Yes 


1 


2 


0 


No 


5 


5 


6 


n = 


6 


7 


6 










OLDER STUDENTS 








Yes 


5 


9 


2 


No 


4 


4 


8 


n = 


10 


13 


10 
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TABLE 13 

HOW IMPORTANT ARE DSES SOCIAL EVENTS? 
(FREQUENCY OF CHECKS) 



ITEM 


SCIENCE 
MAJOR 


ENGINEERING 
MAJOR 


SERVICE 


1ST YEAR STUDENTS 








Not important 


2 


1 


1 


A little important 


0 


0 


2 


Somewhat important 


0 


0 


0 


A lot important 


2 


2 


0 


Very important 


0 


0 


0 


n = 


6 


7 


6 










OLDER STUDENTS 








Not important 


0 


0 


1 


A little important 


2 


0 


1 


Somewhat important 


4 


3 


1 


A lot important 


2 


7 


2 


Very important 


1 


2 


0 


n = 


10 


13 


10 
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Instructional Design and Evaluation Interviews about Quality Instructional Materials 



Creating Quality Instructicnal bfaterials 



Introduction 



The Department of Instructional Design and Evaluation at the 
National Technical Institute for the Deaf (NTID) is composed 
of graphic artists, photographers, media specialists, instructional 
developers and an evaluation specialist. Together, these indi- 
viduals design and develop a wide range of materials for 
instructional purposes such as workbooks, brochures, infor- 
mational flyers, posters, and overheads. 

The National Technical Institute for the Deaf has a 20+ year 
history of producing curricular materials for deaf students. 
Yet, NTID is a relatively young institution and, as such, a 
creative spirit of experimentation predominates as well as an 
on-gcing desire to meet the educational and instructional 
needs of a student population with diverse learning needs. 

Just how students' educational needs are identified, how in- 
structional materials are then developed and designed to meet 
those needs, is a matter of considerable importance to the entire 
Instructional Design and Evaluation Department, particularly 
as these can inform future development activities both at NTID 
and for the education of deaf college students in general. 

Method During the 1989-1990 academic year, each member of the 
Instructional Design and Evaluation Department was asked to 
participate in an interview with the Evaluation Specialist to 
first, define quality as it applies to the type of design or 
development activity engaged in by that person and second, to 
discuss the criteria used by that person in planning and creating 
a high quality instructional material. A third question asked 
individuals to identify an area(s) where they would like to have 
more information or knowledge (for example, specific evalu- 
ation feedback that could more directly inform their work). 

Each member of the Instructional Design and Evaluation De- 
partment was interviewed separately, with the exception of 
two people who were interviewed together. Each interview 
was appro> mately one hour in length. Participants were told 
the purpose and topics of the interview and were encouraged 
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to prepare for the interview in any way that they believed 
would be most helpful to them. They were also encouraged to 
bring copies of completed projects or ones in-process to the 
interview. 

The Evaluation Specialist/interviewer took detailed notes of 
the interview content and then divided the notes into groups 
according to job area. This served to condense the information 
in a way that allowed for individual anonymity. 

People were asked to describe what they do and then to explain 
how they make their decisions. To preserve the flavor of their 
actual responses and the ways in which they were framed and 
organized, the findings are divided into two primary areas: a) 
a narrative portion consisting of comments provided during a 
general discussion of the person's approach to an instructional 
piece and b) a boxed figure containing points made when they 
reviewed specific instructional media. In several cases, the 
content in these two areas overlap. 

Media/Production Specialists 

Media /Production Specialists produce overheads, flyers, dis- 
play pieces, brochures, charts and graphs, slides, newsletters, 
banners and signs, certificates, name tags, T-shirts, and buttons. 
The following are the combined points made by four Media 
Specialists: 

The job needs to be framed in terms of The Five Ws plus H: Who, 
What, When, Where, Why and How. 

Who is the client, audience, and particular media specialist. 

What is the type of product. 

When is it needed? When it will be used, and 

Where? 

Why questions focus on format and production in light of such 
concerns as the likelihood of frequent updates. 
How will it be used - once and then thrown out? Will it be 
completed and returned to us in the mail? Or will it be used 
multiple times? How realistic and adequate is the client's 
conceptualization of the project/product? 

Several constraints on the project or product need to be consid- 
ered along with the five W's and H. Some examples of con- 
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straints are the budget, the number and type of personnel 
working on the project/product, the time of year the request 
was presented (e.g. summer requests may need more time 
because there is less student help available; the same is true for 
quarter breaks). 

The ultimate goal is to be able to take advantage of a wide range 
of options within the constraints you have. The objective 
becomes the encapsulation of the five W's plus H while achieving 
visual simplicity, having narrative brevity while being clever, 
catchy, dynamic and having all elements relevant to the con- 
tent. This means knowing the limitations of the available tools, 
what can and cannot be accomplished, and anticipating as best 
as possible what the finished product will look like. Quality 
starts with the five W's plus H and the conceptualization of the 
design and each subsequent choice made in terms of layout, 
color, paper, and printing. 

Layout Criteria used in judging the quality of a layout include 
balance, density of the text, and the use of visuals to create a 
balanced perspective. The goal is a clean, fresh,crisp, uncrowded 
look. It's important to use an appropriate, readable, legible 
typeface. This requires taking into account such things as 
whether it will need to be read from a distance (a poster) or by 
one person close up (a brochure). Even though balance is an 
objective, visual variety is important. Thus, the goal is to 
achieve proportion among the various visual elements. When 
there is an interruption of one thought or section to another, 
those areas need to be defined. The reader should not have to 
work too hard to understand it or focus on it. The graphic 
elements should focus attention on what most needs the reader's 
or viewer's attention. 

Color. When it comes to color, it is desirable to select contem- 
porary colors and achieve a look that is alive, and not flat. By 
"stretching the colors" it can appear to be a multi-color job. 

Paper. The choice of paper will affect how colors will print. 
Paper texture and color can work together to create a dynamic 
looking product. Added features are the type used, the layout, 
and the use of stretched colors. The quality of the fit of these 
elements depends on the print quality. 



Printing. Depending on the budget and time constraints, and 
the number of copies desired, you may go with photocopying 
or with printing. For print jobs, it's important that the colors 
match the swatches (in other words, the product matches the 
idea of what that product was going to look like). You also want 
even coverage with no show through of the paper. The color 
should be even throughout, from side-to-side and top-to- 
bottom. There should not be any ghosting from the previous 
page. Finally, you want to check to make sure letters (like 'e') 
and knock-outs aren't filling in. 

Before actual production, the mechanicals need to be checked 
to make sure that everything is straight, that all design elements 
are securely fastened, that there is no broken type or uneven 
lines, that there are no missing elements and that there is 
nothing crooked in the paste-up. The mechanicals should 
always be clean and neat without smudges or dirt spots for the 
cameras to pick up. Once printed, you want to make sure that 
the paper is cut evenly and that the paper has been folded well. 

In summary, quality frames the entire product and the full 
process of that product's production. The product should have 
variety and be dynamic, but balanced and with its elements in 
proportion. It should visually attract the reader by being 
inviting and enticing, easy to follow - but not predictable. You 
also want to give the reader some visual breaks. 



Characteristics of a good product 


Characteristics of a poor product 


¥ The type is fine and readable. 

¥ Good use of white space (e.g. 
use of wider gutters or the 
space between columns) and 
line lengths that are visually 
appealing. 

¥ A directional flow of the 
heading, text and visual ele- 


¥ TVpetoc large 

¥ Looks awkward aixl out of balance 

(leeks heavy en the bottom) 
¥ Elements out of proportion 
¥ Items out of visual 9eqjuenae 
¥ Static grcphics 

¥ The type size is too big and it makes 

the layout appear grey 
¥ Things oonpete with one anotter 

for attention cn the page. 



Photographers 



As was true for the Media/Production Specialists, the two 
photographers interviewed gave examples of their products. 
These include such instructional materials as a display, a 
photography show, a slide show, a photo of a faculty member 
for Focus, a photograph that will go into the Mac and become 
part of a newsletter, and an illustration as part of a publication 
where it will be produced as a halftone on a photostat camera 
for reproduction on a photocopy machine. 

Photographic quality can be thought of in terms of the quality 
of the craftsmanship, the presentation, its aesthetic quality and 
how well it communicates, and the kind of sensory response 
that it evokes in the viewer. Quality is very personalized. 
Overall quality achieves a harmony among the above 
elements. 

The quality of the craftsmanshL This involves balance, com- 
position, and energy. It means that there is a dynamism to it. 
For our instructional projects, it involves being appropriate for 
the purpose. It must be technically right. Are people going to 
be drawn to it? It means having the essential elements while 
being innovative and exciting. It requires knowing the media 
- for example, if if s for a slide show, the image needs to be 
simplified so that the viewer will get the information quicker. 
You can ha ve a] 1 of the essential techni cal elements but yet ha ve 
a photograph thafs very boring. Now you get into quality of 
invention and that comes from the quality of the photographer: 
the person's treatment of the seif and the person's treatment of 
people and how much of the self is put into the creation of that 
image. It involves a perfection of one's craft, possessing the 
skills and tapping into an ability that results in a work standing 
out. Yet there's a point where a higher level of quality isn't 
appropriate and is a waste of time. 

The quality of the communication. We don't know why we 
respond the way we do. Something just hits us and a feeling is 
created. It has to do with either the subject matter or the way 
that subject matter has been seen and captured (for example, 
the lighting that tells about texture, the time of day, location.) 
It establishes a mood. It touches many senses and may evoke 
a memory and call up previous life experiences. "Quality of 



communication" is creating a feeling versus just recording an 
image and it brings knowledge, understanding or insight to 
someone. 

Research questions. Are students able to get the information 
that they need? How could they get it better? We need their 
feedback. 

It would be great to have an environment fostered where more 
open conversations occur about our projects from beginning to 
end. We could all offer responses that might be helpful. 



Characteristics of a good photograph. 
(Two exarples of a perscn sitting at a 
ocnputer.) 


Characteristics of a poor photograph 
(fran Navy of cne of cur graduates) 


¥ Consistency between the photos 
throoghout a project /product. 

¥ U9e of lighting (to give a three- 
dimaisicral effect) 

¥ Quality of the preparation (for a 
ptotograph of a person at a 
ocnputer, taking the tine to hide 
the power cords; Getting a 

high 

quality image cn the terminal as 
well as of the person) . 

¥ Getting contrast even when phcr- 
tographing white cn white. Hav- 
ing a full toral range. 

¥ A ocnpositicn where the details 
shew yjp in the highlighted areas 
and in the shadows. 

¥ The manipulaticn of exposure time 
and developrnent, corpressing the 
teres. 


¥ The lighting was flat so the photo 

lacked enerov. 
¥ There was no directionality 
¥ The carposition was not good — 
there was a carputer terminal and 
the light bounced off it and gave a 
feedback. 

¥ The negatives were urcierexposed so 

the prints had no ocntrast . 
¥ No cropping. 
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Instructional Developers 



Existence and performance (adequacy and excellence) are two 
broad benchmarks that may be applied across each of lthe 
following instructional development areas: 

• instructional content 

-composition 
-clarity 

-logical progression and transition 

-standard usage 

-reading level 

-purpose 
-comprehensiveness 

-depth 

-breadth 
-accuracy 

• instructional design 

-learning objectives 
-instructional strategies 

-topic selection and sequencing (macro 

level) 

-chapter /lesson text display features 

-display of facts, concepts, procedures, 
principles 

-text format/displays 
-use of visuals 

-use of mathemagencis Gearner helps) 

-incorporation of practice 
-reference grid 
-reference features 
-appeal / motivation 

-satisfaction 

-elegance 

-efficiency 

-learning objectives/content/testing congruence 

• project management 

-needs assessment 

-instructional need 
-marketability 
-life cycle 
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-per capita cost 
-product specification 
-team roles arid responsibilities 
-communication 

• evaluation 

-product 

-effectiveness 

-performance 

-acquisition of skills 
and knowledge 
-efficiency 
-motivation 
-satisfaction 
-use 
-process 

-satisfaction 
-efficiency 
-time line 

-secondary influence 

-teaching strategies 
-content 

-innovativeness/ creativity 

The first benchmark (existence) simply requires an answer to 
the question, "Was it (inclusion of mathemagenics, satisfaction, 
and so on) part of the product or process?" The next questions 
would be, "Should it have been there?" Performance bench- 
marks ask, "How good was it?" Two levels are then addressed: 
adequacy (minimal) and excellence (the best possible under 
any circumstance). 

Quality, as it pertains to instructional development, moves 
closer to an operational definition by addressing existence and 
performance of instructional content, project management and 
evaluation features and processes. 

Research question: In thinking of hearing and deafness on a 
continuum, what design structures (text and visual) best sup- 
port or provide the most effective learning for profoundly deaf 
students and the other students on the continuum. There may 
be cells with different types of students and the materials best 
suited to their learning. 
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Characteristics of a good design Qiaxacteristics of a poor ctesign 



¥ Has utility , aasssifadJity ard^Rpeal 
¥ Is acmfortable to read ard locks 

inviting. It has a gxd feel about it 
¥ Looks streamlined and is clearand 

arninct 
¥ Uses a readable type face 
¥ SiidTeadings, italics serve to organ- 
ize the material ard learning 
¥ ^fprcpriate language level 
¥ Writing is clear and ocncise 
¥ Use of visuals to show instead of tell 
¥ The visuals enphasize the right 

things and do not distract the reecfer 
¥ Learning is chunked, stqps are bro- 
ken down. 
¥ A variety of elements are used with 
out making the product appear 
ocnplex. 

¥ Visual elements hold the reader s 
attQTticn and reinforce viiat s 
been read 

¥ Consistency of style and presenta- 
tion. No unrelated surprises. 

¥ Has a clear table of acntentsandtes 
ef fecti\ely used color coding ard 
tabs to organize the material . 

¥ It grabs readers' attention, then 
involves them and gets than to ac- 
tively participate in the learning. 

¥ It s a successful learning tool be- 
cause it includes review sections 
and errplcys strategies that reinforce 
and build ipcn the learning. 



¥ Jontains abstract concepts in reed 

of clarificBtim 
¥ Too much infonration broken into 

3mHpartsgi\^aclil±erBd^pear^ 

anoe. 

¥ Lodes like a project that grew ard 

became too ccrrplex. 
¥ There are many inneoessary elements 

used and it ends ip confusing the 

rearfer. 

¥ It has many ccrrplex diagrams and 
looks ssary. 




Artists 



A product can be looked at in terms of three dimensions: 
utility, execution and aesthetics. The sum total of the above 
three areas is quality. Yet, quality is a very personal thing and 
what the client wants can be different from what is personally 
pleasing or satisfying to the artist. In commercial art, if s a 
perspective of how the job is done versus how the product 
emerged. It's almost the same as the difference between 
inductive and deductive logic. Aesthetics can vary from day to 
day and can depend upon the mood of the artist. This is what 
leads to style. For example, depending on the artisf s mood, 
one may select groen over pink for a particular product. That 
in turn may lead to a preference for a particular typeface which 
may suggest a two-color as opposed to a f our-colo r layout and 
so on. Another influence on aesthetics is technical execution. 
Does it attract attention, convey a mood and communicate a 
message. Looking at the execution of the intended message, 
then the relevant criteria for quality would be such questions as 
"was the idea focused and was it communicated successfully - 
- even if the design may not be technically good?" If you look 
at something and a question comes to mind about what the 
artist was trying to do, then something has gotten in the way. 
But if instead the viewer says , "Isn't that interesting, what does 
the artist mean?" then you have communicated something - 
even if they don't know what. If it makes the viewer think 
about the content, if s a good design. If it makes the viewer 
think about the artist, if s not such a good design. 

Technical adequacy and executio: There's a lot of tricks and 
formulas to help an artist sell a product and to commercially 
communicate something. There are many thing you learn in 
design class: spatial relationships, balance, the history of what 
has worked best for certain things, what current elements are 
in vogue, what current elements /style will tell the message 
best, what the market will respond to. This foundation leads to 
preferences and choices regarding typefaces, shapes, sizes and 
ways to illustrate. There are a million ways all of these can be 
combined and a good design depends heavily on making the 
right choices and combinations. But what do you choose and 
how do you put it together? These are quality considerations. 
You want the result to be a consistent, harmonious piece, (how 
the illustration matches with the type and how the various 




elements work together, not compete) that is timely and has a 
feel of having been executed effortlessly but professionally. 
Just some relevant factors are type density, how the different 
messages are brought together and how things are separated 
(the use of space). Graphic design is organizing space - what 
should be left out and what should be put in and how should 
the elements be arranged. A grid is an invisible way of 
organizing elements. How it is all put together, how it commu- 
nicates it' s message within the budget and time constraints, is 
quality. It takes talent, a lot of skill and a knowledge of how to 
utilize your past errors and work around constraints. It also 
has to do with the artist's sense of freedom and flexibility. Does 
the artist have the time to push the elements? To use contrasting 
elements to create drama? (Large against small, black against 
white, patterns against bleakness). Often what you leave out is 
as important as what you leave in. For example, a word that has 
been selectively excluded from a sentence grabs our attention. 

Usefulness, utility, function. This means being concerned 
with picking up the tone of the message and illustrating con- 
sistently with that. It involves being attentive to the clarity of 
the information presented, it's orientation and degree of ambi- 
guity. For example, is it a composition that will confuse the 
viewer if seen from a distance, or just the reverse? A graphic 
solution must begin with a good idea. Then the consideration 
moves to how well it was executed. It involves knowing when 
less is more and when and where to simplify tilings to clarify 
the message. It's getting a feel for the message, knowing the 
audience and communicating to that audience. How you did 
that is by working with design elements, pushing the elements, 
and then refining the design for maximum appeal. While there 
are formulae and techniques, there is always a range of con- 
sistency for the design to fit with. You can float in and out of a 
structure in a way that serves to draw attention and create a 
dynamic illustration. While a grid or a structure can be an 
invisible means of organizing, it should not be constraining. 
Violations of it at times can best point out certain aspects of 
things. 

Aesthetic Where art comes in is in the intuitive decisions the 
artist makes about how far to go. If, however, something is 
distracting, then you apply the reverse process and analyze the 
piece- are the colors working, are there too many elements, is 
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it confusing and the viewer does not know where to look first, are 
the relative sizes of things competing? Is there too much contrast 
or not enough. The bottom line is getting the information across 
so that it's clear and visually appealing. To do that, the designer 
must understand the information , the audience, and have the 
freedom and flexibility to apply art to it. All parts of a design 
contribute equally to the whole product and its success. Art 
becomes, then, the difference between the formula and the expected 
successful piece and a work of appeal and getting the viewer to 
stop, and react, and think and have something in them moved. 
When this happens, then the presentation itself will have a pro- 
fessional quality about it, a seriousness, a caring, and an under- 
standing where the artist comes through. 

To summarize, there are four key points to remember: 

• know your audience 

• know the intended communication (message) 
and select the appropriate elements (type, 
spacing, photos, illustrations, spatial 
relationships, etc.) for that message 

• know when to be general and when to be 
specific (when to be conceptual or literal) 

• strive for balance and appropriate emphasis 

Research questions 

1. Under what circumstances are photographs 
better than graphic illustrations? 

2. What do we miss by not using color? 

3. In what ways do the products work for the 
instructor and the students? In what ways don't 
they? What are we doing right? Is there a better 
or more preferred way? 

4. NTID has its own subculture. What are the visual idioms 
common to NTID students and can we use them? Do they 
structure incoming information in a different way? For 
example, they are more receptive in the periphery of their 
vision than many people and we may be able to use this. 
Start studying this by selecting a small project where the 
students read a sentence and we present it visually with 
three visuals representing one idea (graphics, pictures, 
photos). We can't always repeat success by using what's 
worked in the past and we may be missing some wonderful 
opportunities by prematurely eliminating some ideas. 



Characteristics of a Good Design 


Characteristics of a Poor Design 


V Message is clear (appropriat 
emphasis) 

V Looks dynamic and has energy 

V Caiprehensive, yet timely 

¥ Graphic elenents work well together 
(is clear, sirrple, strong, and in 
balance) 

¥ Involves the viewer (attracts atten 
ticn through size, color, si±>ject 
natter, etc.) 

¥ impropriate elements 

¥ Consistency of thoughts 


V Too ambiguous and communi 
ration is clouded. Lades sinplicity. 

V Too iruch information to take it 

V Ir^picpxlate use of typogic^hy (too 
nary type styles, letter/line 5p=cing) . 

¥ Poor choiae/balance of elements and 
space 

¥ lack of oensistency 



Evaluation Specialist 

There are several types of evaluation products or outcomes, 
and quality considerations differ somewhat among them. 

Evaluation of Instructional Materials and Products 
For quality considerations regarding instructional materials, 
one focuses on aspects of a) the product and b) the process that 
product went through to achieve completion (see "evaluation" 
section under Instructional Developers). 

The Instructional Developers integrate an instructional piece 
and keep in mind many of the criteria mentioned by the artists, 
photographers, and media /production specialists. The Evalu- 
ation Specialist then addresses questions around the use of the 
material. Evaluation quality considerations include such con- 
cerns as: Characteristics of the students using the piece (En- 
glish skills, year of study, etc.), intended use of the material and 
actual use (stand-alone workbook, reference resource, etc.), 
and laser satisfaction (student and instructor). 

Evaluation of Processes 

While the evaluation of instructional materials is a primary 
responsibility of the evaluation specialist, another is the evalu- 
ation of departmental processes (such as judgements about 
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quality). Feedback to the evaluation specialist comes in the 
form of comments, recommendations, etc by the members of 
the Instructional Design and Evaluation Department. Here, 
the focus is on a product, but the evaluation process is that 
product. Put another way, the product emerged or evolved 
from an evaluation process (that is, the process of interviewing 
resulted in this report). Quality considerations for this type of 
evaluation are concerned with the means used to elicit informa- 
tion. For example, the use of good interviewing techniques 
(attending skills, clarification, probing, reviewing, etc.), the 
elimination of distractions (like a tape recorder, lack of privacy, 
etc.), and giving people the chance to review and correct the 
E valuator's interpretation of what was s.~ " /4 Jie latter serving 
as the primary indicator of quality). 

Evalaution of Programs 

Another type of evaluation concerns programming. For ex- 
ample, a summary report of a series of workshops given by 
Training and Development. This type of evaluation can be 
used simultaneously in a formative way (to guide modifica- 
tions in the training programming) and summative manner (to 
demonstrate to NTID executive management that continued 
training in this area is needed). Here, quality concerns go 
beyond a particular product or project and must address such 
concerns as alignment with Institute priorities, the fair repre- 
sentation of faculty/staff needs, and so on. 

Meta Concerns in Evalaution 

Finally, evaluation is also concerned with the patterns of or 
trends in findings over time. No longer product or project 
specific, this type of evaluation addresses broader issues of 
learning. 



Characteristics of Gcxxi Evaluations 


Characteristics of Poor Evaluations 


¥ Uses an appropriate design 

¥ Answers the questions of 
interest to the audience (does 
not ask what no one wants to 
know, does not ask what is 
known already, does not ask 
about the unknowable) 

¥ Has been designed to yield the 
desired information in the 
clearest, simpliest, and most 
economical way possible 

¥ Says only what it legitimately 
can report 

¥ Gives recipients f.eedback 
(negative, positive, or neutral) 
in a manner designed to facili- 
tate improvement and positive 
change 


¥ Used an inappropriate data 
collection and/or data analysis 
strategy 

¥ Reports information that is 
ambiguous or potentially mis- 
leading without appropriate 
qualification 

¥ Claims more than it should 



Conclusion 

1. There is a lot of consistency in definitions of 
what a quality product should look like. 

2. The production of a quality product results from 
a team effort which includes all IDE disciplines. 



Quality definitions are dynamic and, as such, the 
contents of this working paper can be expected to 
change over time. 



Appendix C 



Instructional Material Development for Optical Finishing Technology 
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Optical Finishing Technology Math 1 

Basic Characteristics of Spherical and Cylindrical Lenses 

Student Workbook 

Evaluation Report 

M.J. Scherer, BJ. Siegel, and D.J. Peroni 
Project Description 

The course, Optical Finishing Technology Math 1 is described as follows in 
Undergraduate Courses 1992-3. Rochester Institute of Technolog y (p. 196): 

This course focuses on the rules of transposition, including transposition of lens 
powers. Students learn to apply mathematical functions, solving for binocular 
and monocular P.D.s, near-vision prescriptions, and bifocal segment height and 
inset. The concepts of plus and minus cylinder prescription powers are 
discussed, and definitions and determinations of lens powers from given base 
curves, cross curves, and inside curves are taught. 

Instruction is by classroom lectures and laboratory assignments. To assist 
student learning of the course's concepts, a workbook was developed that would 
define terms and phrases and depict them graphically. The workbook is a 109 page 
document divided into seven units, each approximately 15 pages long and having one 
definition/description or exercise per page. Each definition/description has both a 
verbal and graphic representation of the concept. Most of the odd numbered pages 
are blank pages labeled "Notes" for students to record their own comments about that 
particular concept. The workbook contains both a Table of Contents and an Index. 

Prototype Evaluation Questions 

The objectives faculty have for the workbook is that the descriptions and 
exercises included in it will help students to a) understand characteristics of lenses and 
b) apply their knowledge in the optical finishing laboratory. Key questions for 
students included the following: 

1. Are there definitions/descriptions that are not clear? 

2. Are there illustrations that are not clear? 

3. Are there terms or concepts that are not included that should be? 

4. Are there illustrations that are not included that should be? 

5. Is the workbook easy to use? 

1 
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Evaluation Methodolog y 
Four data sources were selected for addressing the evaluation questions: 

1. Former and current students in the course will be asked to go through 
the workbook and critique it. They will be given copies of the workbook for 
noting desired changes and will have a week to critique it. Then they will 
participate in a group interview with the course instructor, instructional 
developer, and evaluator. 

2. For the lab assignment on using the vertometer, former students will 
come into the laboratory and assist current students in learning to ir.e the 
vertometer. At the end of the lab, both former and current students participate 
in a group interview about the usefulness of the workbook in successfully 
completing this lab assignment. The group interview will involve the course 
instructor, instructional developer, and evaluator. 

3. Student final exams will be analyzed for areas revealing weak conceptual 
understanding. This will be done by the course instructor. 

4. For each lecture and laboratory session, the instructor will note directly 
in the workbook areas of particular strength and weakness. 

The instructor's notes, notes from the student interviews and workbooks, and the 
exam data will be used by the instructor and developer to go through the Workbook 
page-by-page and make revisions. 



Evaluation Results 
Interviews with Selected Former and Current Students 

Six students participated in the group interview. Each had gone through the 
workbook and noted directly on the workbook's pages where they would like to see 
modifications. The interview proceeded by going through the workbook page by page 
and asking students to share their comments, which were recorded by the evaluator. 
After the interview, each student gave their workbook to the course instructor. At a 
subsequent meeting between the instructor and course developer, the students' 
comments and notes were analyzed for improving the workbook. 

Student feedback at this phase of the evaluation was very specific to 
definitions/descriptions in the workbook. The attached pages provide examples. In 
general, students perceived a need in many places for more examples and additional 
graphics. 
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SPHERICAL LENSES 



Spherical lenses are the most basic type of ophthalmic 
lenses. 

The surface of the spherical lens has the same curvature in 
all meridians. It has curves similar to a ball. 

Light rays that pass through a sphere lens will converge to a 
sing'e image point. 



For example: 



Image 
Point 




Positive A positive sphere lens resembles the surfaces of a ball. 
Sphere 

Lens For example: 






i 



i 



A positive sphere lens is used for the correction of 
Hyperopia (farsighted). 



Hyperopia - Light rays 
focus behind the retina. 



U*5 
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Correction of Hyperopia 
with positive sphere lens. 
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Calculating the inside curve of a semi-finished lens 

F T Power (total) 
F, « Front Curve 
F 2 « Inside Curve 



Rx +1.00 

BC +4.00 




For the lens above the inside curve of -3.00 was calculated 
like this: 

F, -+4.00D 
F, .+1.00D 



+4.00 + F, 



Ft 
+1.00 

+1.00 -4.00 
-3.00 
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Thin Lens Formula 

Usually the Thin Lens Formula is written 

D, + D 2 «D 
The D means Diopter Value. 



Sometimes the formula is written 

F, + F 2 * F T 
The F means Focal value 

Both formulas are used the same way. 

XL 
18 1 



Unit of measur-em.en-1- 
US£ci 4o Measure 
pother ©£ cu lerl5. 



Minus 

Sphere 

Lens 



Minus sphere lenses (negative sphere lenses) are similar to 
positive sphere lenses, except light rays that pass through a 
minus sphere lens will diverge. 



This means that there is no real image point after light rays pass 
through the lens. 

However an imaginary (virtual) image point is produced in front of 



♦he lens. 



Virtual Image 




Real Image 



Image Point 
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Image Point 
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A minus sphere lens also resembles the surface of a ball. 




Minus sphere lenses are used for the correction of 
myopia (nearsightedness). 



Focus of eye 
with Myopia 




Retina 
i 



Correction of Myopia with 
a Minus Sphere lens 
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Laboratory Observation on the Use of the Vertometer 

The feedback from this phase of the evaluation suggested ways to improve both the 
workbook and the classroom instruction given to students. It became most clear 
during this evaluation that the workbook in its present form could not serve as a self- 
instructional tool and is dependent on the instructor's explanation. Regarding the 
section in the workbook on vertometers, students agreed on the following: 

There is a need to clearly show which way to turn the power drum. 

More examples and practice with power crosses ( Fl, F2, FT). 

There is still confusion with axis orientation. 

Use arrows to show how to rotate to + or - . 

In summary, it became evident that there is a need for more examples and practice 
tests. Students recommend that each of the seven units include a practice test at the 
end. 

Observations from Students' Final Exam Performance 

This evaluation resulted in the identification of additional modifications. For example, 
some definitions and word choices need clarification. One lens diagram was hard for 
most students to identify. Nov/ the following will be added, "a lens that is thinner in 
the center is a negative sphere lens/' 

Students didn't make the connection between power crosses and curves. An 
illustration will be added to show the relationship. 

In teaching the course, more attention needs to be given to math with plus and minus 
numbers. 

Students need to be helped to make conceptual connections, 

A good idea wouid be to append a basic review of mathematical principles. That way 
it becomes a reference for students as they progress through the workbook. 

A discussion will occur between the instructor and the Math Department so that the 
latter is informed of the difficulties students are having in the mathematical 
understanding of particular concepts. 
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Summary of Implications/Recommendations 



Product 

The evaluation helped identify areas in need of modification or augmentation. The 
evaluation also revealed the need to help students transfer their knowledge into other 
coursework. 

Overall, the workbook has been determined to be a valuable resource to students and 
all of the former students wished that the workbook had been available for their use. 

Process 

The process involved feedback from former and current students as well as the on- 
going observations of the faculty member involved. Everyone worked together as a 
team to come up with better ways of illustrating a concept, deriving examples, and 
creating practice questions. Thus, the evaluation process was both student- and 
instructor-driven. 
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Optical Finishing Technology 



Evaluation Report 

15 October 1990 
Marcia Scherer, Evaluation Specialist 

The project involved the creation of a workbook for students beginning their 
major in optical finishing that would serve as a dictionary of terms used in that field 
and which they would encounter frequently during their course of study here. The 
product is a 9C-page document divided into 18 sections that are contained in a 3- 
ring looseleaf binder. Each of the 18 sections is approximately 5 pages in length and 
has an average of four definitions per page. Each word or term is defined verbally 
and many have accompanying illustrations to aid learning. The definitions appear 
on the odd-number pages; the even-number pages contain blank blocks across from 
each definition so that the student can add his/her own notes about that particular 
word or ter n. 

Method 

Four students who are in the last year of their optical finishing program were 
asked to review the workbook. These senior students were familiar with each word 
or phrase in the dictionary and, thus, were not provided with the workbook. They 
had seen the workbook, however, from newer students and asked to have copies. 
In exchange for complimentary copies of the workbook, they agreed to participate 
in an evaluation addressing the following topics: 

1. List any definition that is not clear. 

2. Note any illustrations that are not clear. 

3. List any words or terms not included that you feel should be added to 
the workbook. 

4. Indicate any sentences that are unclear, confusing, or are unnecessary. 

5. Indicate where illustrations should be added and what they should 
include. 

6. Provide feedback on how easy the dictionary is to use. 

7. Provide feedback on the areas opposite the definitions that has been 
included for note taking and indicate the usefulness of this idea. 

The four students were given one week to review the workbook and answer the 
above seven questions. On Thursday, October 11, they meet as a group with the 
course instructor, instructional developer for this project, and the evaluation 
specialist and provided their feedback. 

Finding s 

The students were urtaninous in their view of the workbook as being a very 
important and useful learning resource and said it was easy to use. They felt that 
the blank areas for note taking were a good idea and, in fact, had recorded their 



own notes in several of them. For many definitions, they had indicated ways in 
which they could be modified, expanded, or otherwise made more clear. 

By and large, the most significant feedback involved the need for an 
illustration for each definition. E/en such "Lens Imperfection" words as scratch, 
chip, and crack were determined to be in need of illustration as well as such abstract 
concepts as "photochromic" (meaning a lens that darkens in color when exposed to 
sunlight or ultra violet light). When asked how they would like an illustration to 
look, they were able to provide either a graphic or visual description. The drawings 
below are how two students would illustrate "photochromic." 



PHOTOCHROMIC 



/ 



I n 
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A lens that darkens in color when exposed to 
sunlight or ultra violet light. 

• Tims photochromic lenses are light indoors and 
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darken in the sun. 




Conclusions 

The methodology employed in this evaluation proved to be fruitful in two 
ways: First, important feedback was obtained from "experienced users" on ways to 
improve individual definitions and the workbook in general and, second, an 
exploratory technique designed to elicit deaf students' visual interpretations of 
narrative material was attempted and found to be successful. The students' ideas 
for expanding and clarifying some of the definitions, as well as their ideas for 
illustrations to accompany definitions, are in the process of being implemented by 

the instructional developer. 



